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Abstract 



In experiments, the dynamical behavior of systems is reflected in time 
series. Due to the finiteness of the observational data set it is not possible 
to reconstruct the invariant measure up to arbitrarily fine resolution and ar- 
bitrarily high embedding dimension. These restrictions limit our ability to 
distinguish between signals generated by different systems, such as regular, 
chaotic or stochastic ones, when analyzed from a time series point of view. 
We propose to classify the signal behavior, without referring to any specific 
model, as stochastic or deterministic on a certain scale of the resolution e, 
according to the dependence of the (e, T)-entropy, /i(e, r), and of the finite 
size Lyapunov exponent, A(e), on e. 
PACS:05.45.Tp, 05.45.-a 
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I. INTRODUCTION 



It is a long debated question if and by what means we can distinguish whether an ob- 
served irregular signal is deterministically chaotic or stochastic If the signal was 
obtained by iterating a certain model on a computer, we can give a definite answer, because 
we know the law which generated the signal. 

In the case of time series recorded from experimental measurements, we are in a totally 
different situation. Indeed, in most of cases, there is no unique model of the "system" which 
produced the data. Moreover, we will see that knowing the character of the model might 
not be an adequate answer to the question of the character of the signal. For example, 
data of Brownian motion can be modeled by a deterministic regular process as well as by a 
deterministic chaotic or stochastic process, as we will show in section 

In principle, if we were able to determine the maximum Lyapunov exponent (A) or the 
Kolmogorov-Sinai entropy per unit time (hKs) of a data sequence, we would know, with 
no uncertainty, whether the sequence is generated by a deterministic law (in which case 
A, hxs < oo) or by a stochastic law (in which case A, hj^s — ^ oo). 

In spite of their conceptual relevance, there are evident practical problems with such quan- 
tities that are defined as infinite time averages taken in the limit of arbitrary fine resolution, 
since, typically, we have access only to a finite (and often very limited) range of scales. In 
order to cope with these limitations, in this paper we make use of the "finite size Lyapunov 
exponent" (FSLE) |^, a variant of the maximum Lyapunov exponent, and the (e, r)-entropy 
per unit time p|- |10|] , a generalization of the Kolmogorov-Sinai entropy per unit time. Basi- 
cally, while for evaluating A and hxs one has to detect the properties of a system with infinite 
resolution, for determining the FSLE, A(e), or the (e, r) -entropy per unit time, h{e,T), the 
investigation on the system is performed at a finite scale e, i.e. with a finite resolution. A(e) 
gives us the average exponential rate of the divergence between close (on scale e) trajectories 
of a system, and h{e, r) is the average rate of information needed for prediction. If properly 

defined, h{e,T) ''f^ h^s and A(e) ^—^ A, if A > 0. Thus, if we have the possibility of 
determining the behavior of A(e) or h{e, r) for arbitrarily small scales, as pointed out above, 
we could answer the original question about the character (deterministic or stochastic) of 
the law that generated the recorded signal. 

However, the limits of infinite time and resolution, besides being unattainable when deal- 
ing with experimental data, may also result to be physically uninteresting. As a matter 
of fact, it is now clear that the maximum Lyapunov exponent and the Kolmogorov-Sinai 
entropy are not completely satisfactory for a proper characterization of the many faces of 
complexity and predictability of nontrivial systems, such as, for instance, intermittent sys- 
tems [|ri|] or systems with many degrees of freedom |0,|l2|. For example, in the case of the 
maximum Lyapunov exponent, one has to consider infinitesimal perturbations, i.e. infinites- 
imally close trajectories resp. infinite resolution. In systems with many degrees of freedom 
(e.g. turbulence) an infinitesimal perturbation means, from a physical point of view, that 
the differences 5xk = x'f. — Xk of the components, x'f. and Xk, of the initially close state vectors 
x' and X, have to be much smaller than the typical values Xk of the variables Xk- If the x^-s 
take very different values, then the concept of infinitesimal perturbation becomes physically 
unimportant, in the case one is interested only in the evolution of the components with the 
largest typical values |1T3|J7|| (e.g., the large scales in a turbulent motion). 
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Taking into account all the limitations mentioned above, in particular the practical im- 
possibility to reach arbitrarily fine resolution, we propose a different point of view on the 
distinction between chaos and noise. Neither it relies on a particular model for a given data 
set nor it ignores the fact that the character of a signal may depend on the resolution of 
the observation. Indeed /i(e, r) (or equivalently A(e)) usually displays different behaviors 
as the range of scales is varied. According to these different behaviors, as it will become 
clear through the paper, one can define a notion of deterministic and stochastic behavior, 
respectively, on a certain range of scales. 

In section ^ we recall the definitions of the (e, r)-entropy and the finite size Lyapunov 
exponent. In section |T|we discuss how one can consistently classify the stochastic or chaotic 
character of a signal by using the information theoretic concepts as the (e, r)-entropy and 
the redundancy and compare our approach with previous attempts. In section |IV| we discuss 
some examples showing that systems at the opposite in the realm of complexity can give 
similar results when analyzed from a time series point of view. Section [V| is devoted to a 
critical discussion of some recent, intriguing and (sometimes) controversial results for data 
analysis of "microscopic" chaos, in particular we comment the point of view to be adopted 
in interpreting the result of sec. In section the reader finds some remarks on non 
trivial behaviors of high dimensional systems. Section |Vli| summarizes and concludes the 
paper. 



II. TWO CONCEPTS FOR A RESOLUTION DEPENDENT TIME SERIES 

ANALYSIS 

A. (e, r)-entropy and redundancy 

In this section we recall the definition of the (e, r)-entropy discussing its numerical com- 
putation, the possible technical problems as well as its properties. 

We start with a continuous (in time) variable x(t) G IR'^, which represents the state 
of a (i-dimensional system, we discretize the time by introducing a time interval r and we 
consider the new variable 

X("^)(t) = (x(t),x(t + r),...,x(t + mr-r)). (1) 

Of course X(™^(t) G JR^^ and it corresponds to the discretized trajectory in a time interval 
T = mr. 

Usually, in data analysis, the space where the state vectors of the system live is not known. 
Mostly, only a scalar variable u{t) can be measured. In these cases one considers vectors 
X(™)(t) = (u{t),u{t + r), . . . , u{t + mr — r)), that live in IR*" and allow a reconstruction 
of the original phase space, known as delay embedding in the literature M,|T3[. It can be 
viewed as a special case of ([l|). 

We introduce now a partition of the phase space IR"^, using cells of length e in each of 
the d directions. Since the region where a bounded motion evolves contains a finite number 
of cells, each X'^"^^(t) can be coded into a word of length m, out of a finite alphabet: 

XM(t) iy-(e, t) = (z(e, t), z(e, t + r), . . . , z(e, t + mr - r)) , (2) 
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where i{e,t + jr) labels the cell in IR"^ containing x(t + jr). From the time evolution of 
X^™''(t) one obtains, under the hypothesis of stationarity, the probabilities P(Vr™'(e)) of the 
admissible words {W"^{e)}. We can now introduce the (e, r)-entropy per unit time, h{e,T) 



h^{e,T) = -[Hm+i{e,T) - Hm{e,T)] (3) 
r 

/i(e,r) = lim hm{e,T) = - lim — if^(e,r), (4) 

m^oo \ ' / ^ m^oo 777, 

where Hm is the block entropy of block length m: 

^^n.(e,r) = - Y: m™(e))lnP(l^™(e)). (5) 

{VK™(e)} 

For sake of simplicity, we ignored the dependence on details of the partition. For a more 
rigorous definition one has to take into account all partitions with elements of size smaller 
than e, and then define /i(e, r) by the infimum over all these partitions (see e.g. |TD|). In 



numerical calculations we circumvent this difficulty by using coverings instead of partitions 
(see below). 

A concept which is complementary to the e-entropy is the e-redundancy (see e.g. |]I6[ ) 



which measures the amount of uncertainty on future observations which can be removed by 
the knowledge of the past, namely: 

rm{e, r) = - [Hi{e, r) - (/f™+i(e, r) - H^{e, r, ))] , 

T 

where Hi{e) estimates the uncertainty of the single outcome of the measurement, i.e. ne- 
glecting possible correlations in the signal. Alternatively, we can write the redundancy in 
the form 

r.m(e, r) = -Hi{e, r) - hm{e, r) , (6) 
r 

which emphasizes the complementarity between redundancy and entropy. If the data are 
totally independent, i.e. IID, one has ifm(e, t) = mHi{e) and, therefore, rm(e, r) = 0. On 
the opposite side, in the case of a periodic signal the redundancy is maximal rm(e, r) = 
Hiie,T)/T. 

The Kolmogorov-Sinai entropy, hxs, is obtained by taking the limit e, r — 0: 

= linilini/i(e,r). (7) 

The KS-entropy is a dynamical invariant, i.e. it is independent of the used state represen- 
tation (|I]), while this is not the case for the e-entropy (^). To simplify the notation we drop 
the T dependence in the following, apart from cases in which the r dependency is explicitly 
considered as in section IV. 

In a genuine deterministic chaotic system one has < h^s < oo {h^s = for a regular 
motion), while for a random process h^s = oo. The entropies H„i{e) were above introduced 
using a partition and the usual Shannon entropy, however it is possible to arrive at the same 
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notion starting from other entropy-like quantities, which are more suitable for numerical 



investigations. Following Cohen and Procaccia [17| one can estimate Hm{e) as follows. 
Given a signal composed of successive records and the embedding variable X^™'^ let us 
introduce the quantities: 



^ N -m ^ ... 



i^5:e(e - |X(-)(^r) - X(-)(jr)|) , (8) 



then the block entropy i/m(e) is given by 



^^H^) = -(]^i:innr(e). (9) 



In practice n^J^\e) is an approximation of P{W"^{e)). From the numerical point of view the 



even more suited quantities are the correlation entropies []T8|,|19 

H'i' (^) = - In E (^)] < (^) ' (10) 

where one approximates the Shannon entropy by the Renyi entropy of order q = 2. 

In the determination of h^s by data analysis, one has to consider some subtle points 
(see for a detailed discussion). 



Let us just make some remarks about the general problems in the computation of the 
Kolmogorov- Sinai entropy from a time series of a deterministic system. The first point is 
the value of the embedding dimension m. Let us assume that the information dimension 
of the attractor of the deterministic system is D. In order to be able to observe a finite 
entropy, m has to be larger than D, since the behavior of the entropies in the limit e ^ is 

hm{e) = const + 0(e) > hxs , (11) 

provided m > D The second relevant point is the fact that the saturation, i.e. the 

regime where the entropy /im(e) does not depend on the length scale e, can be observed only 
on length scales smaller than some eu- Thus it is possible to distinguish a deterministic signal 
from a random one only for e < £„. Due to the finiteness of the data set there is a lower 
scale ei below which no information can be extracted from the data. Taking into account 
the number of points of the series, A^, it is possible to give the following relation between 
the embedding dimension, the K S-entropy, the information dimension and the saturation 
range e^/ei P2[: 



J < (iVe-"^"'^^^) ^ , (12) 



where and ei are the upper and lower bounds of the interval of scales at which the 
deterministic character of a deterministic signal shows up. Note that this relation does not 
determine e^. For more details see ^2 



If m is not large enough and/or e is not small enough one can obtain misleading results, e.g. 
see sec. 0. 
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The e-entropy h{e, r) is well defined also for stochastic processes. Its dependence on e 



can give some insight in the underlying stochastic process |10]. In the case of finite r it is 



possible to define a saturation range: below some length scale eu(r) we have 

hm{€) = const — In e + 0{e) (13) 

But, the limit r ^ will lead to e„ 0, thus the saturation will disappear. As it was 
shown in [|T0|, for some stochastic processes it is possible to give an explicit expression of 
h{e, t) in this limit. For instance, in the case of a stationary Gaussian process with spectrum 
S{u!) oc cu"^ one has 

lim h{e, r) ~ ^ , (14) 



+0 



the same scaling behavior is also expected for Brownian motion |jTO[ . It can be recovered by 
looking at h{e, r) in a certain (e, r) region. See Ref. [|I^] for a detailed derivation of We 
have to stress that the behavior predicted by eq. ([T^) may be difficult to be experimentally 



observed due to problems related to the choice of r (see sect. 



B. Finite Size Lyapunov Exponent 

The Finite Size Lyapunov Exponent (FSLE) was originally introduced in the context of 
the predictability problem in fully developed turbulence ||^. Such an indicator, as it will 
become clear below, is for some aspects the dynamical systems counterpart of the e-entropy. 

The basic idea of the FSLE is to define a growth rate for different sizes of the distance 
between a reference and a perturbed trajectory. In the following we discuss how the FSLE 
can be computed, by assuming to know the evolution equations. The generalization to data 



analysis is obtained following the usual ideas of "embedology" [jl5[. First one has to define a 
norm to measure the distance e(t) = | |5x(t) 1 1 between a reference and a perturbed trajectory. 
In finite dimensional systems the maximum Lyapunov exponent is independent of the used 
norm. But when one considers finite perturbations there could be a dependence on the 
norm (as for infinite dimensional systems). Having defined the norm, one has to introduce 
a series of thresholds starting from a very small one eo, e.g., = r^eo {n = 1, ...,P), 
and to measure the "doubling time" (Tj.(e„)) at different thresholds. Tr(e„) is the time a 
perturbation of size e„ takes to grow up to the next threshold, e„+i. The threshold rate r 
should not be taken too large, in order to avoid the error to grow through different scales. 
On the other hand, r cannot be too close to one, because otherwise the doubling time would 
be of the order of the time step in the integration (sampling time in data analysis) affecting 
the statistics. Typically, one uses r = 2 or r = ^/2. For simplicity is called "doubling 
time" even if r 7^ 2. 

The doubhng times Tr(en) are obtained by following the evolution of the distance | |5x(i) 1 1 
from its initial value emin <^ eo up to the largest threshold ep. Knowing the evolution 
equations, this is obtained by integrating the two trajectories of the system starting at an 
initial distance emin- In general, one must choose emin -C eo? in order to allow the direction 
of the initial perturbation to align with the most unstable direction in the phase-space. 
Moreover, one must pay attention to keep ep < esaturation, so that all the thresholds can be 
attained {esaturation is the typical distance of two uncorrelated trajectories). 
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The evolution of the error from the initial value emin to the largest threshold ep carries 
out a single error-doubling experiment. At this point the model trajectory is rescaled at 
a distance e^m with respect to the true one, and another experiment starts out. After M 
error-doubling experiments, we can estimate the expectation value of some quantity A as: 

(^)e = ^E^»- (15) 
-'^ i=l 

This is not the same as taking the time average because each error doubling experiment may 
take a different time from the others. For the doubling time we have 

for details, see [j^. The method described above assumes that the distance between the two 
trajectories is continuous in time. This is not the case for maps or for discrete sampling 
in time, thus the method has to be slightly modified. In this case Tr{en) is defined as the 
minimum time at which e{Tr) > re„, and now we have [0 



It is worth to note that the computation of the FSLE is not more expensive than the 
computation of the Lyapunov exponent by the standard algorithm. One has simply to 
integrate two copies of the system and this can be done also for very complex simulations. 

One can expect that in systems with only one positive Lyapunov exponent, one has 
A(e) ~ h{e), see Ref. for details. Additionally it is shown in Ref. how it is possible 



to use the FSLE for characterizing the predictability (also from the data analysis point of 
view) of systems containing a slow component and a fast one. Let us comment on some 
advantages of the FSLE with respect to the (e, r)-entropy. For the FSLE it is not necessary 
to introduce an e-partition and, most important, at variance with the (e, r)-entropy, the 
algorithmic procedure automatically finds the "proper time"; so that it is not necessary to 
decide on the right sampling time and to test the convergence at varying the words block 
size A^. This point will be discussed in section [IV C\ 



III. CLASSIFICATION BY e-DEPENDENCE 

In the previous section we discussed the e-entropy and the FSLE as tools to characterize 
dynamical processes. Let us re-examine the question of distinguishing chaos and noise posed 
in the Introduction. Equations (|ll|) and ([T3|) allow us to make rigorous statements about 
the behavior of the entropy in the limit e — > 0. Then the behavior of the redundancy can be 
determined by using the relation to the entropy, given by (P) if we take into account that 
Hi{e) oc — In e for continuous valued non-periodic process. Both the behavior of the entropy 
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and the redundancy are summarized in the following table 0. 



deterministic (m > D) 


stochastic 


r^(e) oo 


hm{e) oo 


chaotic 


non-chaotic 


white noise 


colored noise 


lim„^oo/im(e) > 


lim^^oo hjn{e) = 




r„(e) > 



The behavior of the FSLE in the limit e — > is similar to that of the e-entropy, h{e). It 
is worth noting that the FSLE defined through the doubling times (see sect. II B|) is zero 
also if A < 0. 

In all practical situations we have only a finite amount of data. Let us assume we have 
embedded the time series in a m-dimensional space, e.g. by time delay embedding. Then 
one can relate to this set of points an empirical measure fi* 

1 ^ 

^*P^{m)^ = ^5(X('") -X^'^Hir)) . (18) 
1=1 

This empirical measure /x* approximates the true measure only on length scales larger than 
a finite length scale ei. This means that we cannot perform the limit e — 0. Of course on 
a finite scale ei, both entropy and redundancy are always finite, therefore we are unable to 
decide which will reach infinity for e — 0. But we can define stochastic and deterministic 
behavior of a time series at the length scale dependence of the entropy and the redundancy. 
Fig. D shows the typical behavior of the entropy hm{e) and the redundancy rm(e) in case of 
a deterministic model (2- dimensional chaotic map) and a stochastic model (autoregressive 
model of first order). 

For a time series long enough, a "typical" system can show a saturation range for both 
the entropy and the redundancy. For decreasing length scales e with e < e„ one observes the 
following behaviors 



deterministic 


stochastic 


r„(e) oc - Ine 


/im(e) oc — Ine 


hm{^) ~ const 


rm(e) ~ const 



In addition, as far as stochastic behaviors are concerned, the e-entropy can exhibit power 
laws on large scales, e.g. in the case of diffusion eq. (p!^ (see section IV and Ref. |]I0| for 
further details). 

If on some range of length scales either the entropy /im(e) or the redundancy rm(e) is a 
constant, we call the signal deterministic or stochastic on these length scales, respectively. 
Thus we have a practical tool to classify the character of a signal as deterministic or stochastic 
without referring to a model, and we are no longer obliged to answer the metaphysical 
question, whether the system which produced the data was a deterministic or a stochastic 
one. 

Moreover, from this point of view, we are now able to give the notion of noisy chaos a clear 
meaning: chaotic scaling on large scales, stochastic scaling on small scales. We have also the 
notion of chaotic noise, namely stochastic scaling on large scales and deterministic scaling 
on small scales. These notions will become clear with the examples in the following sections. 
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The presented method for distinguishing between chaos and noise is a refinement and 
generahzation of one of the first discussed methods to approach this problem: estimating 
the correlation dimension and taking a finite value as a sign for the deterministic nature of 



the signal [Q. The main criticism to this approach is based on the work of Osborne and 
Provenzale 0, where they claimed that stochastic systems with a power-law spectrum will 
produce time series which exhibit a finite correlation dimension. A detailed discussion of 
this problem is beyond the scope of this paper, but a main step to clarify the problem was 
taken by Theiler [^. Firstly, he noted that the discussed signals were non- stationary and 
highly correlated with correlation times of the order of the length of the time series. From 
a conservative point of view one has to stop at this point with any attempt to calculate 
dimensions or entropies. If one proceeds nonetheless, Theiler showed that the result will 
depend on the number of data points and the length scale. If one has a sufficient number of 
data points one will encounter also for this kind of signals the embedding dimension which 
leads to the typical behavior as given in ( ]TB| ) for the entropy. Moreover, if one uses the usual 
time delay embedding like (|^) in contrast to and [^, the result depends strongly on the 
chosen delay time. 

We are aware that there are a lot of other attempts to distinguish chaos from noise 
discussed in the literature. They are based on the difference in the predictability using 
prediction algorithms rather than the estimating the entropy or they relate determinism 
to the smoothness of the signal PJ2^. All these methods have in common that one has to 
choose a certain length scale e and a particular embedding dimension m. Thus they could, 
in principle, shed light at the interesting crossover scenarios we are going to describe in the 
next chapter. 



IV. DIFFICULTIES IN THE DISTINCTION BETWEEN CHAOS AND NOISE: 

EXAMPLES 

In this section we analyze in a detailed way some examples which illustrate how subtle 
the transition from the large scales behavior to the small scales one can be; and thus the 
difficulties arising in distinguishing, only from data analysis, a genuine deterministic chaotic 
system from one with intrinsic randomness. 



A. The diffusive regime 

We first discuss the problems, due to the finite resolution, which one can have in analyzing 
experimental data. We consider the following map which generates a diffusive behavior on 



the large scales [p^]: 



Xt+i = [xt] + F{xt-[xt]), (19) 
where [xt] indicates the integer part of Xt and F{y) is given by: 

w,A_/(2 + A)y ifi/e[0,l/2[ 

""^y^ \ (2 + A)y-(l + A) iiy G]l/2,1]. ^^^^ 
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The maximum Lyapunov exponent A can be obtained immediately: A = In with F' = 
dF/dy =2 + A. Therefore one expects the following scenario for h{e) (and for A(e)): 

h{e) ^ A for e < 1, (21) 



h{e) oc ^ for e > 1, (22) 



e2 



where D is the diffusion coefficient, i.e. 

{{xt - xof) ^2 Dt for large t. (23) 

Fig.s ^ and ^ show A(e) and h{e) respectively. Let us briefly comment on a technical aspect. 
The numerical computation of A(e) does not present any particular difficulties; on the other 
hand the results for h{e) depend on the used sampling time r. This can be appreciated by 
looking at Fig. 25b of the Gaspard and Wang review |T^, where the power law behavior ( p2[ ) 
in the diffusive region is obtained only if one considers the envelope of hm{e,T) evaluated 
for different values of r; while looking at a single r one has a rather not conclusive result. 
This is due to the fact that, at variance with the FSLE, when computing h{e, r) one has to 
consider very large m, in order to obtain a good convergence for Hm{^) — -f^m-i(e)- 

Because of the diffusive behavior, a simple dimensional argument shows that, by sampling 
the system every elementary time step, a good convergence holds for m > /D. Thus, for 
e = 10 and typical values of the diffusion coefficient D ~ 10^^, one has to consider enormous 
block size. A possible way out of this computational difficulty may be the following. We 
call / (= 1) the length of the interval [0, 1] where F{y) is defined; if we adopt a coarse- 
grained description on a scale e = I, i.e. we follow the evolution of the integer part of Xt, 



the dynamical system ( p!9D is well described by means of a random walk [^, with given 
probability po that in a time step s (= 1) the integer part of x does not change: [xt+s] = [xt], 
and probabilities p± that [xt+s] = [xt] ± 1. A diffusive behavior means that the probability 
for a changing of [xt] by ±k in n elementary time steps is given by p8 




g-(fcV2an)(«V«) 

P{k, n) ^ \l -, (24) 

with a a function of po- Now we increase the graining and identify all the x values in a cell 
of size e = L, with L an integer multiple of I. If we observe the coarse-grained state of the 
system only every r(> s) time steps and we define the variables k and n, such that kl ~ kL 
and ns ^ nr, we can write 

-(fcV2an)(LVr) [77 

P{kn)^- . , (25) 

y/ ZTcan V T 

for the probability of finding the system k L-cells apart after n r-intervals. Thus, by choos- 
ing L'^/t = I'^/s, we expect that the sequences generated by checking the system either 
on a scale L every r steps or on a scale / every elementary time step s, have the same 
statistics, in particular the same entropy, as a signature of a diffusive behavior. Note that 
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if liiiim^oo -f^m(e, T = e^)/m is constant at varying e, as we found numerically, then the 
e-entropy per unit time limm^oo -f^m(e, T)/mr goes like 

Since the equality L'^/t = / s is assured by the choice e = L = ■yl and r = 
7^s, one can expect that, for a diffusion process, the following scaling relation hold: 
limm^oo Hm{€, t) / 171 = limm-»oo Hm{l^, 7^t) / m, with 7 an arbitrary scahng parameter. This 
scaling relation allows us to see why the power law behavior (0) is expected to be valid gen- 
erally for the Brownian motion. Indeed, if we choose 7 = 1/e we have Hm{^, t) = Hm{^, t/c^) 
and, finally, taking the limit r — > the e-entropy is given by: 

^ |g„,.(l.r/eVg.„(l.xA^)| ^ co„str/^-^0(r^) ^ 1 

which is (|14D- Note that the first equality in ( p6D has been obtained by a Taylor expansion 
around r = 0, and by noting that h{l, 0) = otherwise the entropy for unit time will be 
infinite at finite e which is impossible. 



B. Finite resolution effects 

We consider now a stochastic system, namely a map with dynamical noise 

xt+i = [xt] +G{xt- [xt]) + (Tr]t, (27) 

where G{y) is shown in Fig. ^ and rjt is a noise with uniform distribution in the interval 
[— 1, 1], and no correlation in time. As it can be seen from Fig. ^ the new map G{y) is a 
piecewise linear map which approximates the map F{y). When dC/dy < 1, as is the case 
we consider, the map (p7D, in the absence of noise, gives a non-chaotic time evolution. 
Now one can compare the chaotic case, i.e. eq. ([T9|) with the approximated map (|27|) with 



noise. For example let us start with the computation of the finite size Lyapunov exponent 
for the two cases. Of course from a data analysis point of view we have to compute the 
FSLE by reconstructing the dynamics by embedding. However, for this example we are 
interested only in discussing the resolution effects. Therefore we compute the FSLE directly 
by integrating the evolution equations for two (initially) very close trajectories, in the case of 
noisy maps using two different realizations of the noise. In Fig. Q we show A(e) versus e both 
for the chaotic ([19|) and the noisy (|27|) map. As one can see the two curves are practically 
indistinguishable in the region e > a. The differences appear only at very small scales e < a 
where one has a A(e) which grow with e for the noisy case, remaining at the same value for 
the chaotic deterministic chase. 

Both the FSLE and the (e, r)-entropy analysis show that we can distinguish three differ- 
ent regimes observing the dynamics of ( P7[ ) on different length scales. On the large length 
scales e > 1 we observe diffusive behavior in both models. On length scales a < e < 1 both 
models show chaotic deterministic behavior, because the entropy and the FSLE is indepen- 
dent of e and larger than zero. Finally on the smallest length scales e < a we see stochastic 



behavior for the system (pTl) while the system (|T9|) still shows chaotic behavior. 
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C. Effects of finite block length 



In the previous section we discussed the difficuhies arising in classifying a signal as 
chaotic or stochastic because of the impossibility to reach an arbitrary fine resolution. Here 
we investigate the reasons, which make it difficult to distinguish a stochastic behavior from 
a deterministic non-chaotic one. In particular, we show that a non-chaotic deterministic 
system may produce a signal practically indistinguishable from a stochastic one, provided 
its phase space dimension is large enough. 

The simplest way to generate a non-chaotic (regular) signal having statistical properties 
similar to a stochastic one is by considering the Fourier expansion of a random signal 0. 
One can consider the following signal: 



M 



x{t) =Y,Xo^sm{n,t + ^,) (28) 



i=l 



where the frequencies are equispaced discrete frequencies, i.e. fli = Qq + iAfl, the phases 
(pi are random variables uniformly distributed in [0, 27r] and the coefficient Xoi are chosen 
in such a way to have a definite power spectrum, e.g. a power law spectrum, which is a 



common characteristic of many natural signals. Of course (pS]) can be considered as the 
Fourier expansion of a stochastic signal only if one consider a set of 2M points such that 
MAQ = vr/At, where At is the sampling time 0. Time series as ( p8D have been used to 
claim that suitable stochastic signals may display a finite correlation dimension P,^, see 
the discussion in sect. |T1|. 

Here we adopt a slightly different point of view. The signal (|28|) can also be considered 
as the displacement of a harmonic oscillator linearly coupled to other harmonic oscillators. 
Indeed, it is well known since long times that a large ensemble of harmonic oscillators can 
originate stochastic-like behaviors I^Oj. In particular, we refer to [Q, where it was proved 
that an impurity of mass /i linearly coupled to a one-dimensional equal mass, fio, chain of M 
oscillators coupled by a nearest-neighbor harmonic interaction, in the limit of /i ^ /iq and 
of infinite oscillators (M — > oo), undergoes a Brownian motion. Practically our observable 
is given by the sum of harmonic oscillations as in eq. (|28|) , where the frequencies Qi have 
been derived in the limit fio/fi -C 1 by Cukier and Mazur [Q. The phases (pi are chosen 
as uniformly distributed random variables in [0, 2n] and the amplitudes Xoi are chosen as 
follows: 

Xoi = Cnr' (29) 

where the C is an arbitrary constant and the Q dependence is just to obtain a diffusive-like 
behavior. Note that for a signal of length 2M the random phases and the Xq^'s represent a 
initial condition of the M oscillators, because their phase space is 2M-dimensional. 

In Fig ^ we show an output of the signal (pS]) and, for comparison, in Fig^D we also show 
an artificial continuous time Brownian motion obtained integrating the following equation 

^ = (30) 

where ^ [t) is a Gaussian white noise, produced by a random number generator (the variance 
of the process is chosen as to mimic that obtained by (^)) . Because the random number 
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generator uses a high entropic one- dimensional deterministic map, this is an example for a 
high entropic low dimensional system, which produces stochastic behavior. As it is possible 
to see the two signals appears to be very similar already at a first sight. 

Now it is important to stress that if M < oo the signals obtained according to (^) cannot 
develop a true Brownian motion especially if one is interested in long time series. Indeed 
for a long enough record one should be able to recognize the regularities in the trajectory 
of x{t). However, even if the time record is long enough, in order to give a definite answer 
about the value of the entropy one also needs very large embedding dimensions. The basic 
fact is that deterministic behavior can be observed only, if the embedding dimension m is 
larger than the dimension of the manifold where the motion takes place, which is M for 
M harmonic oscillators. This means that although the entropy hxs is zero, the conditional 
entropies, t) = (-?/m+i(e) — Hm{e))/T, for finite m are nonzero and maybe even slowly 
decreasing for m > M. Moreover, one can encounter some quasi-convergences with respect 
to m for m < M , if r is large enough, i.e. the entropy can seem to be independent of m, 
e.g. see Fig. |[ 

In Fig. l^-b we show the e-entropy, calculated by the Grassberger-Procaccia method 
||18|| . The deterministic signal (^) and the stochastic one (^) produce indeed very similar 
results. Note that we calculated /i^^(e,r) instead of /i^^(e, r) because it is used more often 
due to better statistics in most cases. However, in Fig. | one can see, that on the relevant 
length scales both entropies are almost equal. 



As in Refs. [[10],^^ we have considered different time delays, r, in computing the e-entropy 
because of the problems discussed in the previous section. The power law behavior for 
the e-entropy is finally obtained only as the envelope of different computations with different 
delay times. The results for the FSLE calculated from the time series are shown in Fig. |^. 
Both the e-entropy and the FSLE display the 1/e^ behavior, which denotes that the signals 
can be classified as Brownian motion . It is worth noting that the FSLE computed from 
the time record is not too sensitive on the choice of the delay time r and the embedding 
dimension m. 

From this simple example is it easy to understand that the impossibility of reaching high 
enough embedding dimensions severely limits our ability to make definite statements about 
the "true" character of the system which generated a given time series as well as the already 
analyzed problem of the lack of resolution. 



V. SOME REMARKS ON A RECENT DEBATE ABOUT "MICROSCOPIC" 

CHAOS 



The issue of the detection of "microscopic" 



some attention |l32| , |33[| after a work of Gaspard et al. |^ 



chaos by data analysis has recently received 
Gaspard et al., from an entropic 



analysis of an ingenious experiment on the position of a Brownian particle in a liquid, claim 
to give an empirical evidence for microscopic chaos, i.e.: they claim to give evidence that 
the diffusive behavior observed for a Brownian particle is the consequence of chaos on the 
molecular scale. Their work can be briefly summarized as follows: from a long 1.5 ■ 10^ 
data) record on the position of a Brownian particle they compute the e-entropy with the 
Cohen-Procaccia [|1^ method, described in section ||, from which they obtain: 



13 



Me) ~ ^ (31) 



where D is the diffusion coefficient. Then they assume that the system is deterministic 
and therefore, because of the inequality h{e > 0) < hxS) they conclude that the system is 
chaotic. However, from the results presented in the previous sections we can understand 
that their result does not give a direct evidence that the system is deterministic and chaotic. 
Indeed, the power law (PT|) can be produced in different ways: 



1. a genuine chaotic system with diffusive behavior (as the map (^) of sect. [IV A[ ); 

2. a non chaotic system with some noise, as the map (|27|); 

3. a genuine Brownian system, as (|3 



4. a deterministic linear non chaotic system with many degrees of freedom (as the example 



5. "complicated" non chaotic system as the Ehrenfest wind-tree model where a particle 
diffuses in a plane due to collisions with randomly placed, fixed oriented square scatters 
(as discussed by Cohen et al. in their comment to Gaspard et al. ||31|| ). 

It seems to us that the very weak points of Gaspard et al. are: 

a) the explicit assumption that the system is deterministic 

b) the neglecting of the limited number of data points and, therefore, of both limitations 

in the resolution and the block length. 

The point (a) is crucial, without this assumption (even with an enormous data set) it is not 
possible to distinguish between 1) and 2). One has to say that in the cases 4) and 5) at least 
in principle it is possible to understand that the systems are "trivial" (i.e. not chaotic) but 
for this one has to use a very huge number of data. For example Cohen et al. estimated 
that in order to distinguish between 1) and 5) using realistic parameters of a typical liquid, 
the number of data points required has to be at least ~ 10^^. Let us remind that Gaspard 
et al. ^T|] used ~ 1.5 ■ 10^ points! 

It seems to us that the distinction between chaos and noise makes sense only in very 
peculiar cases, e.g. very low dimensional systems. Nevertheless even in such a case an 
entropic analysis can be unable to recognize the "true" character of the system due to the 
lack of resolution. This is particularly evident in the comparison between the diffusive map 
(pop and the noisy map (pTj), if one only observes at scales e> a. According to the pragmatic 



proposal of classification discussed in sect. |ITl| one has that for o" < e < 1 both the system 



(pop and (p7D, in spite of their "true" character can be classified as chaotic, while for e > 1 
both can be considered as stochastic. 

In this respect, the problem of the lack of resolution is even more severe for high entropic 
systems. One can roughly estimate the critical e (e^), below which the saturation can be 
observed, to be e„ oc ex]i{—hKs)- Indeed exp(/ix5) estimates the number of symbols, i.e. 
cells of the partition required to reconstruct the dynamics. Therefore, in the Brownian 
motion studied by Gaspard et al. ||^, where the i^'S'-entropy is expected to be proportional 



to the number of molecules present in the fluid, the possible small e„ is pushed on scales far 
from being reachable with the finest experimental resolution available. 
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VI. DISCUSSIONS 



Here we briefly review two examples, studied in details in P^ , pB| , showing that high- 
dimensional systems can display non-trivial behaviors at varying the resolution scales. We 
believe this discussion can be useful in furtherly clarify the subtle aspects of the distinction 
between stochastic and deterministic behaviors in dynamical systems. 

Olbrich et al. [0] analyzed an open flow system described by unidirectionally coupled 
map lattice: 

Xj{t + 1) = (1 - a)/(xj+i(t)) + axj{t) (32) 

where j = 1,. . . ,N denotes the site of a lattice of size N, t the discrete time and a the 
coupling strength. A detailed numerical study (also supported by analytical arguments) of 
the e-entropy h{e) at different e, in the limit of small coupling, gives the following scale- 
dependent scenario: for 1 > e > cr there is a plateau h{e) = As where is the Lyapunov 
exponent of the single map x{t + 1) = f{x{t)). For o" > e > cr^ another plateau appears at 
h{e) ~ 2Xs, and so on for a""^ > e > cr" one has h{e) ~ nXg- Similar results hold for the 
correlation dimension which increases step by step as the resolution increases, showing that 
the high-dimensionality of the system becomes evident only as e — > 0. 

Therefore one has a strong evidence that the dynamics at different scales is basically ruled 
by a hierarchy of low-dimensional systems whose "effective" dimension ne//(e) increase as e 
decreases: 



neff{e) 



ln(l/6) 
ln(l/a) 



(33) 



where [. . .] indicates the integer part. In addition, for a resolution larger than e, it is possible 
to find a suitable low- dimensional noisy system (depending on e) which is able to mimic xi{t) 
given by eq. (P^. It is interesting to note that, looking at h{e) on an extended range of 
values of e, one observes for e > 

hie) ~ In ^ (34) 

i.e. the typical behavior of a stochastic process. Of course for e < one has to realize 
that the system is deterministic and h{e) ~ A^A^. Even if this study mainly concerns 
the small unidirectional coupling, the diffusive and strong coupling case deserves further 
analysis, it represents a first step toward the understanding of the issue of data analysis of 
high-dimensional systems. 

Let us now briefly discuss the issue of the macroscopic chaos, i.e. the hydro-dynamical- 
like irregular behavior of some global observable, with typical times much longer than the 
times related to the evolution of the single (microscopic) elements composing a certain 
system. This interesting kind of dynamical behavior has been studied in some recent works 
|35| , |36[| for globally coupled maps evolving according to the equation: 



N 

x^it + 1) = (1 - a)UxM) + T7 E faMt)) (35) 
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where is the total number of elements and fa is a nonlinear function depending on a 
parameter a. Cencini et al. [^] (see also Shibata and Kaneko for a related work) 
studied the behavior of a global variable (i.e. the center of mass) using the FSLE analysis. 
Their results can be summarized as follows: 



at small e (^ l/y/N) one recovers the "microscopic" Lyapunov exponent, i.e. A(e) 
A. 



^micro 



at large e (^ l/yN) one observe another plateau (corresponding to what we can call 
the "macroscopic" Lyapunov exponent) A(e) ~ )^macro which can be much smaller than 
the microscopic one. 



The above results suggest that at a coarse-grained level, i.e. e ^ l/y/N, the system can 
be described by an "effective" hydro-dynamical equation (which in some cases can be low- 
dimensional), while the "true" high-dimensional character appears only at very high resolu- 
tion, i.e. 

e < ^ O ^ 



NJ 

The presence of two plateaux for the FSLE at different length scales is present in generic 
systems with slow and fast dynamics The interesting fact is that in systems like ( pSf ) 
the two temporal scales are generated by the dynamics itself. 

Let us stress that the behavior h{e) = const at small e and h{e) decreasing for larger e is 
not a peculiarity of the diffusive map (pOt). In typical high- dimensional chaotic systems one 
has h{e) = hxs ~ 0{N) for e < ec (where N is the number of degrees of freedom and ec ^ 
as iV — >■ oo) while for e > Cc, h{e) decreases (often with a power law). From this point of 
view, the fact that in certain stochastic processes h{e) ~ e~" can be indeed extremely useful 
for modeling such high-dimensional systems. As a relevant example we just mention the 
fully developed turbulence which is a very high- dimensional system whose non infinitesimal 
(the so-called inertial range) properties can be successfully mimicked in terms of a suitable 



stochastic process |37 



VII. CONCLUSIONS 

We have shown how an entropic analysis at different resolution scales (in terms of e- 
entropy and Finite Size Lyapunov Exponent) of a given data record allows us for a classifi- 
cation of the stochastic or chaotic character of a signal. 

In practice, without any reference to a particular model, one can define the notion of deter- 
ministic or chaotic behavior of a system on a certain range of scales. In our examples we 
show that, according to the pragmatic classification proposed in Sect. Ill, one can consider 
(on a certain resolution) a system as random or deterministic independently from its "true" 
nature. At first glance this can appear disturbing, however, if one adopts a non "metaphys- 
ical" point of view there is an advantage in the freedom of modeling the behavior of the 
system at least if one is interested on a certain (non infinitesimal) coarse-graining property. 
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FIG. 1. (a) /irra(e) -dashed - and rm(e) - solid - for the Henon map with the standard parameters 
(a = 1.5 and b = 0.3), with m = 2, . . . , 9 and (b) the same for an AR(1) process, with m = 1, . . . , 5 
and fixed r. 
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FIG. 2. The map F{x) (^) for A = .4 is shown with superimposed the approximating (regular) 
map G{x) p7|) obtained by using 40 intervals of slope 0. 
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FIG. 3. A(e) versus e obtained with the map F{y) ( pO| ) with A = .4 (o) and with the noisy 
(regular) map (□) (|27| ) with 10000 intervals of slope .9 with a = 10^^. The straight lines indicates 



the Lyapunov exponent A = In 2.4 and the diffusive behavior A(e) 
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FIG. 4. (e, r)-entropy for the noisy (□) and the chaotic maps (o) with the same parameters as 
Fig. |3|, the encoding method is explained in the text. The straight lines indicates the ii'S'-entropy 
hxs = A = In 2.4 and the diffusive behavior h{e) ~ e~^. The region e < a has not be explored for 
computational costs. 
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FIG. 5. (a) Time record obtained from eq. ( p8| ) with the frequencies chosen as discussed in 
the text with M = 10^, C = .005 and fio/fj. = 10~^, the numerically computed diffusion constant 
is ~ 0.007. The length of the data set is 10^, and the data are sampled with At = 0.02. (b) 
Time record obtained from an artificial Brownian motion ( ^0[ ) with the same value of the diffusion 
constant as in (a). 
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FIG. 6. Dependence from the embedding dimension of the e-entropy calculated with the Grass- 
berger-Procaccia algorithm using at e = 1.85 from the time series shown in Fig. ^a. The horizontal 
line only indicates a possible numerically evaluated value for the saturation of the entropy. 
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(a) (b) 

FIG. 7. e-entropy calculated with the Grassberger-Procaccia algorithm using using 10^ points 
from the time series shown in Fig. |5[ We show the results for embedding dimension m = 50. The 
two straight-lines show the D/e^ behavior. 




FIG. 8. e-entropy estimated by the Cohen-Procaccia algorithm {q = 1) compared to the en- 
tropies calculated by the Grassberger-Procaccia algorithm {q = 2) for m = 20 and r = 0.2. Let us 
stress that the behavior below e = 0.2 is essentially a finite sample effect. 
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FIG. 9. Finite size Lyapunov-Exponent from a artificial Brownian motion (right) and for the 
time series ( |2^ ) (left). Embedding dimension m = 20. Theiler window 1000, i.e. we consider only 
neighbors which are distant in time more than 1000 sampling times. The straight lines show the 
behavior D/e'^, where D = 0.007 is the diffusion constant. The analysis has been performed on 
10^ points. Different computation by changing r and m have lead to the same result. 
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